Old wisdom said: more parameters than data points means memorization and poor generalization. In practice, very large models generalize better than smaller ones.

Two reasons:

• Implicit bias of gradient descent — even when infinitely many functions could fit the training data, SGD naturally gravitates toward smoother, lower-norm solutions rather than memorizing noise. This is a tendency, not a guarantee, and not yet fully understood theoretically.
• Task complexity — a model like Llama 405B has ~15 trillion tokens vs 405 billion parameters, so it isn't even overparameterized in the raw statistical sense. The patterns in language (grammar, facts, reasoning, causality) are deep enough to fully utilize the model's capacity.

A model that memorizes can't compress efficiently — memorization stores every example separately, while generalization means learning underlying structure. Neural networks are fundamentally compression engines: predicting the next token forces the model to compress the statistical structure of language into its weights.

Early LLMs like GPT-3 scaled parameters much faster than training data, leaving many parameters undertrained (only ~1.7 tokens per parameter).

DeepMind's Chinchilla paper showed that for a fixed compute budget, the optimal allocation is roughly 20 training tokens per parameter — scaling both together rather than prioritizing model size.

However, 20:1 is compute-optimal, not deployment-optimal. Modern models deliberately exceed it:

• Llama 1 65B: ~21 tokens/param (near Chinchilla-optimal)
• Llama 3 405B: ~37 tokens/param
• Llama 3 8B: ~1,875 tokens/param

Meta intentionally over-trains smaller models beyond the Chinchilla point, spending more compute at training time to produce models that are cheaper to serve at inference. The loss continues to decrease well past the 20:1 ratio — the optimal ratio for deployment depends on expected inference demand, not just training compute.

Some capabilities — reasoning, coding, in-context learning — don't appear gradually. They seem to switch on once the model reaches sufficient scale, even though the underlying loss improves smoothly the entire time.

Analogy: water heating up — temperature rises smoothly, but boiling happens at a threshold.

Caveat: there is genuine debate about whether emergence is a real discontinuity or partly a measurement artifact. Studies using finer-grained metrics (rather than binary task pass/fail) show much more gradual improvement, suggesting the apparent sudden jump may partly reflect coarse evaluation.

Scaling laws and emergence are not in conflict: smooth improvements in loss can produce nonlinear jumps in task performance once the model crosses a threshold of representational capacity.

Modern AI breakthroughs require enormous compute and are driven by industry (GPT, Llama, AlphaFold), unlike earlier progress which came largely from academia. The transformation may compress a century-scale shift (like the industrial revolution) into a few decades.

On risks: AI has capability but no intrinsic desires. The primary concerns are misuse — misinformation, job displacement, and crime. A pragmatic regulatory response: treat AI-assisted crime as an aggravating circumstance, similar to the use of a weapon.